fix(strix): isolate direct-OpenAI fallback from foreign ambient API base - #1323
fix(strix): isolate direct-OpenAI fallback from foreign ambient API base#1323seonghobae wants to merge 1 commit into
Conversation
In nvidia_nim or openrouter modes the contracted final openai-direct fallback inherited LLM_API_BASE pointing at the primary provider's gateway; that gateway answers an OpenAI-keyed request with a literal '404 page not found', so NIM rate-limit storms always exhausted every fallback and failed closed. resolved_llm_api_base_for_model now returns an empty base for explicit direct-OpenAI models, letting litellm use its default api.openai.com endpoint exactly like direct primaries, while GitHub Models and same-provider fallback routing is unchanged. RED->GREEN covered by a function-execution regression contract pinning: openai-direct/openai_direct isolation under an NVIDIA ambient base, unchanged NVIDIA-base inheritance for non-OpenAI models, and unchanged dedicated-base routing for GitHub Models fallbacks.
|
Warning Review limit reachedNext included review available in 11 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| if is_explicit_openai_model "$model"; then | ||
| # Cross-provider fallback: an explicit direct-OpenAI model must never | ||
| # inherit another provider's ambient API base. In nvidia_nim or | ||
| # openrouter modes LLM_API_BASE_FILE points at that provider's gateway, | ||
| # which answers an OpenAI-keyed chat request with a literal | ||
| # "404 page not found". Returning empty lets litellm use its default | ||
| # https://api.openai.com/v1 endpoint, exactly like openai_direct | ||
| # primaries, while run_strix_once already swaps in the dedicated | ||
| # STRIX_OPENAI_FALLBACK_KEY for authentication. | ||
| return 0 | ||
| fi |
There was a problem hiding this comment.
📝 Info: Ambient base now ignored for direct-OpenAI even as primary
The early return in resolved_llm_api_base_for_model fires for any openai_direct/* or openai-direct/* model, not only fallbacks. A direct-OpenAI model configured as the STRIX_LLM primary with a custom LLM_API_BASE (e.g. an OpenAI-compatible proxy) would now have that base silently discarded and default to api.openai.com. Current org configs use openai_direct only as a fallback, so no live caller is affected.
Was this helpful? React with 👍 or 👎 to provide feedback.
|
Superseded by replacement PR #1331, which re-uploaded the direct-OpenAI fallback API-base fix on current main. |
1 participant
Problem
Every Orgmetra Strix scan in
nvidia_nimmode that exhausted NIM retries (shared org key → 429 storms) then hit the contracted final fallbackopenai-direct/gpt-5.6-lunaand failed with a literal404 page not found. Evidence: Orgmetra strix run 32782227098 attempt 2 — primarynemotron-3-super-120b-a12bRateLimitError ×3, fallbackllama-3.3-nemotron-super-49b-v1.5failure signal, fallbackopenai-direct/gpt-5.4/5.6-lunaLLM CONNECTION FAILED (404), gate fail-closed with zero report artifacts.Root cause
resolved_llm_api_base_for_modelonly overrides the API base for GitHub Models models. Explicit direct-OpenAI fallbacks fell through to the ambientLLM_API_BASE_FILE, which in nvidia_nim/openrouter modes points at the primary provider's gateway.run_strix_oncecorrectly swaps inSTRIX_OPENAI_FALLBACK_KEYbut posts to the NVIDIA/OpenRouter gateway anyway → Go-style404 page not found.Fix
Early return (empty base) for
openai_direct/* | openai-direct/*so litellm uses its defaulthttps://api.openai.com/v1, identical to how direct-OpenAI primaries already route. Key swap at line 2549 is unchanged; no secret or review-agent key scheme touched.RED→GREEN evidence
New function-execution regression contract
tests/test_strix_openai_fallback_base_isolation.py:Existing gate contracts pass locally: nim-not-found, backend-unavailable-after-exempted-finding, model-behavior-error suites (28 tests OK).
Scope
Orgmetra-facing Strix lane only; no review-agent credential changes.